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ABSTRACT 


This thesis seeks to identify factors affecting the probability of selection of a 
Surface Warfare Officer (SWO) to Executive Officer (XO) in the U.S. Navy. Selections 
to XO are made by a board that meets annually. Because a candidate is considered for 
selection in up to three consecutive boards, the possible outcomes in this process are 
selection to XO in one of three annual boards, failure to be selected to XO in the third 
board, or attrition from the process between boards. Using data on the board’s selections 
over a three-year period (2002-2004) a hazards-based logistic regression model is 
developed to estimate the probabilities associated’ with a candidate’s disposition based 
on his or her career profile. The model confirms that a candidate’s recent fitness and 
evaluation report (FITREP) is the single-most-important factor affecting selection. 
Additionally, officers who have completed a tour in Washington D.C. or at the Bureau of 
Naval Personnel have higher probabilities of selection than do those who have completed 
other shore tours. But when an officer receives a poor FITREP, the probability of 
selection is low, regardless of other factors. A nonparametric statistical analysis is used 


to confirm these findings. 
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EXECUTIVE SUMMARY 


The United States Navy Surface Warfare Officer (SWO) community selects 
officers for Executive Officer (XO) by means of an administrative screening board. This 
board convenes annually to select officers from a pool of eligible candidates. Officers 
are eligible for selection to XO within a three-year time window: They may be selected 
on their first, second, or third look; or they may not be selected for XO during their 
career. Conversely, the selection board reviews candidates of whom the majority are 


drawn from three successive promotion year groups (PYGs). 


This thesis examines the career profiles of officers from three PYGs (2002 
through 2004) to identify factors that dispose a candidate either positively or negatively 
to selection to XO. The disposition of an officer in the selection process can be 
represented as one of the following six time-ordered outcomes: (1) selection on first 
review; (2) attrition after first review but before second review; (3) selection on second 
review; (4) attrition after second review but before third review; (5) selection on third 
review; and (6) not selected for XO. Data from three selection boards, which convened 
in 2002, 2003, and 2004, are analyzed using a probability model that incorporates 
explanatory variables, and that recognizes the time-sequential nature of the selection 
process. The model allows for the fact that only one of the cohorts (PYG 02) had 
completed the full three-year selection process; those not selected in the other two 
cohorts (PYG 03 and PYG 04) remained eligible for selection in a future board (in 2005 
or 2006). The model also adjusts for attrition that occurred between selection boards, 
which was not directly captured by the data. Additionally, nonparametric statistical 
techniques are used to explore relationships between the explanatory variables and the 


probability of selection. . 


Explanatory variables derived from the career profiles of officers in the PYG 02, 
PYG 03, and PYG 04 cohorts were used to model the probabilities that an officer would 
progress to the next step of the time-sequential selection process. These variables include 
ratings from the officers’ fitness and evaluation reports (FITREPs) at O3 (pay grade), 


whether the officers served a tour of duty in Washington D.C. or at the Bureau of Naval 


XVil 


Personnel, and on which coast (East, West, or both) the officer served during his or her 
department head tour. Other variables that were considered include whether an officer is 
nuclear-trained, has completed a Master’s Degree, is prior enlisted, and has completed 


Joint Professional Military Education (JPME). 


Several explanatory variables emerge as important in modeling an officer’s 
probability of selection for XO. The most prominent of these is the officer’s FITREP 
rating at the O3 pay grade. Having a good FITREP does not guarantee selection to XO, 
but a poor FITREP is a difficult obstacle for a candidate to overcome. Other factors in an 
officer’s career profile also contribute significantly to the probability model, but none 
carries more weight than the FITREP rating. Nonparametric statistical tests also confirm 
the finding that the O3 score is the most influential attribute leading to an officer’s 


selection or non-selection to XO. 


XVili 


I. INTRODUCTION 


The Surface Warfare Officer (SWO) community is responsible for the manning of 
all US Navy surface ships. It manages the surface community, distributes its officers in 
accordance with the mission of Navy Personnel Command (NPC) PERS-4 (Career 
Management Department), and is responsible for all surface-warfare-related manpower 
issues. PERS-4 is directed by a Navy flag officer and is administratively and 
operationally in charge of the SWO community (PERS-41) for all assignment and 
management issues. PERS-41 is directed by a Navy Captain and is responsible for the 
assignment of officers, placement of officers (ships manning), and the management of the 
community (number of officers). For the purposes of this thesis, the masculine pronoun 


“he” is used, but SWOs can be male or female. 


The SWO community is part of the Unrestricted Line Officer (URL) community. 
Officers in the Navy are categorized into one of five general types: URL, restricted line 
(RL), staff corps, limited duty officers (LDO), and warrant officers. Each of these is 
described briefly below: 

e Unrestricted Line Officer: As the name suggests, a URL officer is one 
who is not restricted in the performance of his duty. URL officers are 
authorized to lead sailors and command ships, submarines, aviation 
squadrons, special operation, and special warfare units. To command a 
commissioned United States Navy ship, an officer must be a member of 
the URL community and be eligible for command-at-sea (Department of 
the Navy, 1990). Command-at-sea refers to a URL officer who is the 
Commanding Officer (CO) of a deployable ship, aviation squadron, or 


other sea-deployable unit. 


e Restricted Line officer: RL officers support URL officers in activities that 
require specialized training and skills in areas that align with operational 
requirements. The command of an RL officer is restricted to areas directly 


related to the officer’s area of expertise. 


e Staff Corps Officer: Staff corps officers are dedicated to provide support 
services to operational units. Staff corps officers include those in the 


Supply Corps, the Medical Corps (doctors), and the Nurse Corps. 


e Limited Duty Officer: LDOs are either line or staff technical managers 
who are limited to performing duties related to their former enlisted 


ratings, generally up to the department head level. 


e Chief Warrant Officer: CWOs are either line or staff technical specialists 
who are limited to performing duties related to their former enlisted 


ratings. 


As of 2006, the SWO community is a diverse group of 7,522 officers. 
Approximately 85.6 percent are male and 14.4 percent are female. However, from the 
rank of Lieutenant Commander (LCDR) through Admiral (ADM), the community is 
approximately 97.2 percent male and 2.8 percent female (Navy Personnel Command 
PERS-41, 2006). 

A. AT-SEA CAREER PROGRESSION OF A SWO 

As a newly commissioned officer, a SWO starts his career serving as a division 
officer (DIVO) on a ship. He will serve as a division officer for two tours, lead a small 
division of sailors, and report to a Department Head (DH). He will also learn the 
fundamentals of surface warfare and attain qualifications as a SWO during this period. 


The division officer tours are approximately 42 months. 


A SWO will return to sea at approximately the eight-year point and will serve two 
tours as a DH. A department head is in charge of one of the various departments on the 
ship and reports to the Executive Officer (XO). The DH tours are approximately 36 


months. 


Upon completion of the DH tours, screened SWOs will serve a tour as an XO 
afloat. All Navy ships have one XO who is responsible for the administration of the ship. 
The XO reports directly to the CO and is second-in-command of the ship. The length of 


this tour varies, but is typically at least eighteen months. 


Finally, screened officers will serve as a CO afloat. The CO of a ship is 
responsible for all aspects of the ship and reports directly to the group commander. The 
length of this tour varies and is a function of the number of SWOs in a particular year 
group. 

To be assigned as a CO on a commissioned US Navy surface ship, an officer must 
follow a defined career path starting at the rank of Ensign (ENS). The following chart 


illustrates the nominal career path of a SWO in relation to his years of service: 
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Figure 1. SWO Career Path (From: Navy Personnel Command PERS-41 Senior 
SWO Mentoring Brief, 2006). 


The O4 board represents the time in a SWOs career where he will be eligible for 
promotion to the O4 pay grade. The boxes titled “1st Look,” ‘“‘2nd Look,” and “3rd 
Look” represents the times in a SWOs career at which he becomes eligible for 
administrative screening to XO and CO. The administrative screening to XO is 
represented first and is the primary focus of this thesis. The O5 board represents 
the time in a SWOs career where he will be eligible for promotion to the O5 pay 
grade. 
B. FOCUS OF RESEARCH 

This thesis seeks to identify factors in a SWO’s career profile that affect the 
likelihood of his being selected for XO by an administrative selection board. A selection 
board is comprised of people making subjective assessments, within the framework of a 


rules-driven process. 


This thesis focuses on answering the following questions: 


e How important is performance at sea to being selected to Executive 


Officer? 


e What is the probability that a Surface Warfare Officer will be selected for 
Executive Officer by the first selection board, the second selection board, 


the third selection board, or that he will not be selected? 


e Which factors in a Surface Warfare Officer’s career profile improve his 


chances for selection to Executive Officer? 


e Which factors in a Surface Warfare Officer’s career profile reduce his 


chances for selection to Executive Officer? 


With answers to these questions, an assignment officer (detailer) will be able to 


direct the career path of a SWO and best prepare him for selection to XO. 


I. BACKGROUND 


A. LITERATURE REVIEW 

Two previous Naval Postgraduate School Master’s theses consider how graduate 
education affects selection or progression through the use of statistical models to estimate 
probabilities. Fuchs (1996) describes the effect of graduate education on promotion and 
CO/XO screening in the SWO community. Fuchs uses a PROBIT model to quantify the 
effect of graduate education on an officer’s probability of promotion. A PROBIT model 
used the standard normal distribution function to describe the probability of success (e.g., 


promotion) as a function of explanatory variables (Montgomery, 2001). 


Fuchs (1996) finds that graduate education has a positive effect on an officer’s 
probability of promotion. Officers who are selected for and complete graduate education 
are more likely to be successful earlier in their career, and they are more likely to screen 
for career milestones such as commanding officer (CO) and executive officer (XO). An 
important difference between the study by Fuchs and the present thesis is that Fuchs only 
considers the effect of graduate education, while this thesis considers a broader set of 
explanatory variables, such as having completed a Washington, DC or BUPERS tour, and 


whether the officer has completed Joint Professional Military Education (JPME). 


The thesis by Wong (2004) develops a logistic regression model to predict 
whether a student is likely to graduate in a particular course of instruction (language) at 
the Defense Language Institute (DLI) in Monterey, CA. Variables such as gender, 
whether a student’s program of study has been realigned, Defense Language Aptitude 
Battery (DLAB) scores, and armed forces service components are included. Wong’s 
model estimates probabilities of graduation in different language categories. Wong finds 
that higher DLAB scores and not having been realigned are traits that are favorable to 
predicting graduation (Wong, 2004). 

B. SELECTION BOARDS 

Administrative selection boards in the U.S. Navy are convened and executed in 

accordance with a published precept letter. A precept letter, signed by the Commander, 


Navy Personnel Command (CNPC), defines the procedures of the board. Board members 


take an oath to follow the guidance of the precept and to not disclose any of the board’s 
proceedings without approval of the Chief of Naval Personnel. Each board consists of 
one President, a variable number of voting board members, one recorder, and a variable 


number of assistant recorders. 


All board members vote for each candidate by issuing scores that express their 
belief in the suitability of the candidate for the position under consideration. These 
scores assume one of five possible values: 0, 25, 50, 75, 100, where any vote greater than 
zero is considered to be a “yes” vote. After each vote, the number of yes votes and the 
confidence-level percentage are announced and recorded. The board then votes on which 
officers to tentatively select and which officers to remove from further consideration 
based upon the distribution of confidence vote scores. The process is then repeated with 
the remaining records until all selections have been made. The following section 
explains the process in more detail. 

1. Process of SWO XO Selection Boards 

After board members have been sworn in, the candidates’ records are divided 
among the board members for review, with each candidate being reviewed by the 
individual board member to whom his record was assigned. During this review, a board 
member has access to an officer’s Officer Summary Record (OSR), Performance 
Summary Record (PSR), and supplementary information, such as letters that the 
candidate may have submitted to the President of the board. Each board member assigns 
a quantitative “grade” to the record based upon an individual’s traits and qualifications. 
After all records have been reviewed and graded by the assigned board member, the 
board retires to the “tank” for deliberations. A tank is a board space that has chairs with a 
secret hand voting system attached to it, large video monitors that are used to brief 
officer’s records, and an electronic pointing system that is used by board members to 


amplify certain traits in an eligible officer’s record. 


Selection board deliberations occur in one of the three “tanks” at NPC. All 
aspects of their deliberations are held in secrecy unless disclosure is approved by the 


Chief of Naval Personnel. 


During deliberations, the records of officers who are eligible for selection to XO 
are presented on a video monitor by the board member who was assigned to review his 
record. During the briefing process, the assigned board member announces his 
quantitative grade and amplifies any other information about the record that he deems 
relevant. This amplified information may consist of performance trends, qualifications, 
or any other information that is not explicitly excluded by the precept (e.g., marital 
status). Grades must be determined in accordance with the guidance given in the precept. 
After the board member has completed his brief, all board members cast their secret 
confidence vote. Once all votes are recorded, the board recorder announces the number 
of “yes” votes and the overall confidence vote score. This process is repeated until all 


eligible officers’ records have been briefed and votes have been recorded. 


After all eligible officers’ records have been voted and recorded, a scatter-gram is 
displayed on one of the tank’s video monitors, showing the distribution of the eligible 
officers’ scores. The board President announces the number of officers that will be 
selected during the first round of voting, and the board decides on upper and lower cutoff 
scores which are used to select and to remove officers from further consideration 
respectively. The board then enters the “crunch” face in which it deliberates on officers 


whose scores fall between the lower and upper bounds. 


The crunch phase repeats the review-and-voting process for officers who were 
neither accepted nor rejected in the first phase. In the crunch phase each officer’s record 
is reviewed by a different board member. These records are again briefed, voted upon, 
displayed in a scatter-gram, and partitioned into selections, removals, or neither. This 
process is repeated until the number of authorized selections is reached in accordance 
with the precept letter (Navy Personnel Command PERS-480, 2006). 

2. Authorized Selections 

As noted above, the precept letter specifies the authorized number of selections. 
These selections are made from a three pools of candidates defined by their Promotion 
Year Groups (PYGs). The most recent PYG is facing the selection board for the first 
time. Candidates from the second-most recent PYG are facing the selection board for the 
second time, having been passed over for selection the previous year. Candidates from 


the third-most recent PYG are those who were passed over for selection the previous two 
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years. In the 2004 selection board, for example, the largest pool of candidates came from 
PYG 04 (first review), the next-largest pool came from PYG 03 (second review), and the 
smallest pool came from PYG 02 (third and final review). The number of selections 


made by the board is allocated among the three pools of candidates using the following 


algorithm: 
e Step 1: Estimate the size of the current-year PYG after all three board 
reviews have been completed based upon historical continuation rates. 
e Step 2: Determine an “opportunity rate” for each PYG (between 60-75 


percent) and multiply this number by the estimated PYG size (number 


from previous step). 


e Step 3: For the PYG in its first review, multiply the result of Step 2 by .50. 
For the PYG in its second review, multiply the result of Step 2 by .30. For 
the PYG in its third look, multiply the result of Step 2 by .20. 


No direct evidence can be found that the board is tougher or more lenient on 
candidates based upon the number of authorized selections, because the board is formally 
committed to select the most-qualified candidates. However, in PYGs that have fewer 
authorized selections, there is more competition between candidates as the number of 
selections is lower (PERS-41, 2006). 

C. SCOPE AND LIMITATIONS 

This thesis develops a statistical model for estimating the probability that a SWO 
attains a specified outcome in the XO selection process. Possible outcomes are that the 
officer will be selected on his first, second, or third selection board; that the officer will 
not be selected for XO; and that the officer will attrite from the selection process between 
boards. Using data for three cohorts of SWOs, which we label “PYG 02,” “PYG 03,” 
and “PYG 04” to reflect the years in which the officers became eligible for selection to 
executive officer, the study examines the disposition of these officers before the annual 
selection boards convening in April of 2002, 2003, and 2004. These three cohorts 
encompass 590 officers with sufficiently complete data for use in the thesis research. 
Data on these officers were extracted from the Electronic Military Personnel Record 


System (EMPRS) and the SWO detailing office (PERS-41). 
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The disposition of the PYG 03 and PYG 04 cohorts before the 2003 and 2004 
selection boards raises the issue of censored data. A PYG03 officer who was not 
selected in 2004 may or may not be selected in the 2005 board. Similarly, a PYG 04 
officer who was not selected in 2004 may or may not be selected in 2005 or in 2006. The 
PYG 02 cohort does not raise the issue of censoring because all officers’ dispositions 
were Settled in the three selections (2002, 2003, and 2004) that are within the scope of the 


thesis research. An analysis of the data must therefore take censoring into account. 


Based on the data, a statistical model is developed that relates the probability of 
selection to XO to factors in an officer’s career profile. Formally, the selection process is 
governed by Department of Navy policy which states, in part: 

The Department of the Navy is dedicated to equality of treatment an 

opportunity for all personnel without regard to race, creed, color, gender or 

national origin. The Navy strives to maintain a professional working 
environment in which an individual’s race, creed, color, gender, or national 
origin will not limit his or her professional opportunities. Accordingly, within 
this board’s charter to determine those officers who are best and fully 
qualified, you must ensure that officers are not disadvantaged because of their 


race, creed, color, gender, or national origin (Navy Personnel Command, 
2006). 


Because of the clearly-stated Navy policy, this thesis does not consider race, gender, or 
ethnic background of the officers as factors that affect their probability of selection to 


XO. 


It must be recognized that attrition occurs in any personnel system observed for a 
period of years. The rate of attrition of SWOs during the three-year career window in 
which they are eligible for selection to XO is believed to be low. In this thesis attrition is 
defined as an officer leaving the SWO community regardless of whether he leaves the 


Navy or changes his occupational specialty within the Navy. 


The data that were available for conducting the thesis research included officers 
who succeeded before the 2002 and 2003 selection boards, and all officers who were 
considered by the 2004 selection board. Officers in PYG 02 who were not selected in 
2002 and left the SWO community before the 2003 selection board convened were not 
captured in the data. Similarly, officers in PYG 02 or PYG 03 who were not selected in 


2003 and left the SWO community before the 2004 selection board were not captured. 
9 


Because attrition is not reflected the PYGO2 and PYGO3 cohorts, their data is 


incomplete. Analysis of the data must reflect this fact. 
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Hil. DATA AND METHODOLOGY 


A. DATA 

The data in this thesis were obtained from the Surface Warfare Officer 
Distribution Division (PERS-41) at NPC. The data are divided into two parts. The first 
part consists of biographical and career-related information that is used by selection 
board recorders during board preparatory work, board proceedings, and post-board work. 
The second part of the data were supplied by the Officer Assignment Information System 
(OAIS) and EMPRS. 

1, PERS-41 Data 

Each year, PERS-41 develops a database consisting of biographical and 
performance data on all officers who are eligible for selection to XO. These data include 
biographical information such as name, rank, Social Security Number (SSN), PYG, 
career milestones completed (graduate education, JPME), types of ships on which an 
officer served, and information that tracks an individual’s status before the XO selection 
board (e.g. missing information, letters to the board, errors, etc). These data are compiled 
from existing sources and are not made available to voting board members. 

2. Officer Assignment Information System (OAIS) 

OAIS is the computer system used by NPC to issue orders and distribute Navy 
officers. _OAIS records both biographical and performance data. Biographical 
information stored in the system includes data such as name, SSN, and dependent status. 
Performance data stored in the system include information such as promotion 
recommendations from Fitness Report and Counseling Record (FITREP), and officers’ 
histories of assignments. 

3. Electronic Military Personnel Record System (EMPRS) 

EMPRS is the Navy’s electronic military records database. This database 
contains an officer’s career record, including academic transcripts, awards, and copies of 
FITREPs. EMPRS is one of the data systems that drives OAIS, and it is through EMPRS 
that data available in local databases are made available. Due to the sensitivity of data 


obtained from EMPRS, it is restricted to authorized users only. 
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4. Creation of Data Sets for Analysis 

PERS-41 uses a Microsoft Access database tool to merge OAIS and EMPRS data 
into one database which contains biographical and performance history of queried 
officers. 
B. REPRESENTATION OF THE REVIEW PROCESS 

When a SWO becomes eligible for selection for XO, he is assigned to a PYG in 
which he may undergo up to three board reviews in successive years, resulting in either 
his selection for XO or in his failure to be selected for XO through this process. During 
the years between reviews, an officer may attrite from the process for various reasons. 
Nominally, a small but non-negligible number of officers are lost due to attrition. Each 
officer, therefore, may be assigned to one of six possible outcomes, which may be 
represented by assigning to an officer one six ordered values denoted by the variable Y: 

Y= 1: SWO is selected for XO in his first board review. 


Y=2: SWO attrites from the XO selection process after his first board, but 
before his second board. 


Y=3: SWO ris selected for XO in his second board review. 


Y=4: SWO attrites from the XO selection process after his second board, but 
before his third board. 


Y=5: SWO ris selected for XO in his third board review. 


Y=6: SWO completes the process (three board reviews) without being selected 
for XO. 


This process is depicted in Figure 2. 
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Figure 2. Outcomes of the Executive Officer Selection Process 


STATISTICAL METHODOLOGY 

An officer who is at the beginning of the selection process has a set of career 
attributes that may affect his probability of selection to XO over the ensuing three years. 
It is reasonable to treat as a random variable the outcome Y that is realized by an officer 


who undergoes the selection process. In this section a probability model for Y is 


developed. 
1. Estimation of Conditional Probabilities 
Let ¥ =(%,.%552+% x,) denote a vector of explanatory variables for a particular 


officer. The objective of probability modeling can be stated formally as follows: 


estimate the (conditional) probabilities 
PU Xe)HPY Ht x), TH=L 2k s 
where, in the present context, k =6. 
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For a given officer, the process of being reviewed for selection to XO may take as 
long as three years. During this time, information about an officer may also change. For 
example, an officer may receive an adverse promotion recommendation (rating) on a 
FITREP during the three years of the selection process that changes the probabilities of 
subsequent outcomes. The use of time-dependent covariates (Cox and Oakes, 1985) 
reflects the changeability of the explanatory variables with respect to time: 


p(t; x@)) = PY =tlx@)), t=1,2,...,k 


It is of interest to predict Y or to estimate its probabilities both using explanatory 
variables at the time an officer enters the selection process, and using explanatory 
variables that change as the officer continues through the selection process. As an officer 


enters the selection process at timef=1f,, a detailer will want to estimate the probability 


that the officer will be selected in any of the three boards for which he is eligible using 


information that is available at that time: 


P(select | x(t,))= >) p(t; x()) 


t=1,3,5 
As the process evolves, and the officer’s outcome has not yet been resolved either 
through selection or attrition, it is of interest to estimate the probability that the officer 
completes the process at the next stage. If the officer has progressed beyond stage t—1 
of the process, it is known that his final outcome will occur at some stage for later. 


Therefore, it is useful to obtain estimates of the conditional probabilities: 


h(t; x(t) =P =tlY >t-1,x(@)), t=1,2,...,k 


In reliability and survival analysis, the conditional probabilities given by 
h(t; x(t)), t=1,2,...,k constitute what is known as the hazard function of the process. 
Note that at time t=1, h(; x(1))= pd;x()), and at time t=k, h(k; x(k))=1. The 
hazard function can be used to calculate probabilities that an officer attains specified 
outcomes conditional on the evolution of his covariates by making use of the following 


inversion formula: 
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t-l 
pts x(t) =hsx@)x] [L-AGsx(@)), t=12,...4 
jel 
The probability that an officer realizes the outcome Y = t should be understood to 


depend upon the history of his covariates x(t) up to and including time f, although the 


hazard function depends only on the value of the covariates at time f. 


2: Hazard Function and Multinomial Outcomes 

The use of statistical models for the hazard function is a standard technique in the 
analysis of survival and reliability data. Cox and Oakes (1985) give a thorough treatment 
of hazard regression, including the use of explanatory variables to model the probability 
(or density) that an item fails at a particular time, given that it was alive at the 
immediately prior time. In this thesis, “failure” refers to a SWO attaining one of the six 
endpoints defined for the XO selection process timeline, which includes the “desirable” 
outcome of being selected for XO. Cox and Oakes (1985) also treat estimation with 


time-dependent variables, which is relevant to this study. 


Modeling of the hazard function in statistical literature usually deals with 
continuous response variables, such as the time to failure of an entity. In this study, the 
outcome variable Y is discrete, which takes on values in the set {1,2,3,4,5,6}. Popular 
models for continuous hazard functions, such as those based upon an assumption of 
proportional hazards, are not applicable to the discrete case, although a discrete hazards 
model bearing some similarity to the proportional hazards model is introduced in Cox 


and Oakes (1985). 


The modeling strategy used in this thesis entails fitting a series of unrelated 
logistic regressions to the hazard function at each transition of Y. Taken as a whole this is 
referred to as the Multistage Hazards Logistic Regression Model (MHLR). The 
parameters of the MHLR model are estimated from data on the PYG 02, PYG 03, and 
PYG 04 cohorts. Let nm denote the number of observations represented, each 
corresponding to an officer in one of the three cohorts. Each officer has an outcome 


denoted Y, and a vector of covariates x,(t) observed over the time frame of the selection 


L 
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process. For the moment, assume that each Y, is known, and that the data may be 


regarded as a representative sample of the SWO community. 


The hazard function is estimated in stages. At the first stage, (1; x,(1))is 


estimated by examining all n candidates and classifying them either Y=1 or as Y>1. 
This simple, binary classification is modeled using logistic regression. At the second 
stage, h(2;x,(2)) is estimated by examining only those candidates for which Y > 1 is 
true, and a new logistic regression model is fitted based on the classification Y =2 or 
Y>2. This process is continued through stagek—1, where the candidates that are 
examined are those for whom Y >k-—2, and the classification is Y=k-—1 or Y>k-1 
(which is the same as Y=k). After the hazard functions have been estimated, 
probabilities p(t; x(t)) may be estimated by using the inversion formula on the estimated 


hazard functions. 


The following notation explains the MHLR modeling procedure in greater detail. 


Let z denote any number strictly between zero and one, and define the logit function as: 


logit(zz) = log (=) 


(In this thesis, all logarithms are natural logarithms unless indicated otherwise.) The logit 
function is strictly increasing, mapping z ~0 to values that approach negative infinity, 
and z ~1 to values that approach positive infinity. At stage ¢ in hazard modeling, the 
following linear model is used: 
logit(h(@ x) =x'O) B =B + xOB ++ +x, OZ, 

By inverting the logit function, this model is expressed equivalently as 

exp(x'(t) 8.) 
1+exp(x'(t) 8.) 


The logistic regression parameters f are estimated using maximum likelihood. 





h(t; x(t)) = 


Software for logistic regression is widely available in statistical packages such as S-Plus, 


Minitab, SAS, and other products. 


In the MHLR model the hazard-function logistic regressions may use different 


sets of explanatory variables, for which coordinates of the coefficient vectors f, are set 
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equal to zero. The total number of parameters that must be estimated across all stages of 
hazard modeling is given by(k—-1)(p+)D-#{£, ; =9}, where the last quantity is the 
number of parameters that are set equal to zero. If B is the estimated logistic regression 
parameter vector, the estimated hazard function is given by 


exp(x'(t) B,) 
1+exp(x'(1) B) 


The estimated outcome probabilities are obtained from the inversion relationship: 





A(t; x(t) = 


Alt; x(1)) =h(,x@)x] | [1-2 t=1,2,....k 


The MHLR modeling procedure outlined above is a special case of modeling a 
multinomial random variable using explanatory variables. An alternative to the MHLR 
model is the neural-network approach taken by Venables and Ripley (1997), which the 


authors implement as the function multinom in their nnet library for the S-Plus® 





statistical computing language. In mult inom, the outcome probabilities are expressed 








as follows: 
Ns oe REA 
1+) exp(x' B,) 
p(ix)=4 7 
1 
= , t=k 
1+) exp(x' B,) 
j=l 


As in the hazards-based model, the number of parameters in the multinom 
model is(k—1)(p+1). Limitations of multinom are that it does not allow using 
different explanatory variables for each coefficient vector, and it does not accommodate 
time-dependent covariates. As a general multinomial model, mult inom does not have 


the flexibility to handle situations where the outcome is a process that evolves over time. 


Under some circumstances, it is desirable to simplify the hazards-based model so 


that the only time-dependent component in £, is the constant term £,,. The coefficients 


for the explanatory variables remain the same across each of the hazard models, resulting 
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in a complete model that has many fewer (p+k-—1) parameters. This simplified model 
is described in Cox and Oakes (1985) as a discrete-time analog to the continuous-time 
proportional hazards model. Although this model is not explored in this thesis, it merits 
further study. 

3. Censoring and Truncation 

The three years of SWO cohort data that are available raises censoring and 

truncation issues that must be addressed in statistical modeling. Truncation implies 
missing data and censoring implies that a variable can be measured only as belonging to a 
set of possible values that is usually defined with respect to time. To understand these 
issues, each cohort group is considered separately: 

e Officers in the 2002 cohort (PYG 02) were available for observation in all 
three selection boards (2002, 2003, and 2004) for which they were 
eligible. None of their outcomes was censored. Only officers who 
succeeded in the 2002 and 2003 were revealed, while all officers who 
succeeded and failed in the 2004 board were revealed. Officers who 
attrited from the process after failing to be selected in their first or second 
board were truncated from the data set. The 2002 cohort was eligible to 


observe the following values of Y: 1,3,5, and6. 


e Officers in the 2003 cohort (PYG 03) were available for observation in the 
2003 and 2004 boards. Officers who attrited from the process after failing 
the 2003 board (their first board) are not captured in the data. Officers 
who failed the 2004 board (their second board) are captured in the data, 
although it is not known how these officers fared in their third board 
(future event). Officers with Y=2 are truncated from the data, and 
officers who failed their second board are represented by the right- 
censored value Y >3. Therefore, the 2003 cohort was eligible to observe 


the following values of Y: 1,3, and >3 (grouping outcomes 4,5,6). 


e Officers in the 2004 cohort group (PYG 04) were not affected by 
truncation. The outcome of every officer before the 2004 board (their first 


board) is known. Those who failed the first board are represented by the 
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right-censored value Y>1. Therefore, the 2004 cohort was eligible to 


observe either Y= 1 or Y>1. 


The issue of right censoring does not impose difficulties for estimation under the 
MHLR model used in this thesis. At each stage, only officers are used who were eligible 
to observe the value for which the hazard function is being calculated. Right censoring at 
stage f results in a negative answer to the question “did the officer take on the value Y =+ 


given that Y >t—-1?” 


Truncation, however, does present difficulties which preclude fitting the MHLR 
model in a stage-wise manner. In the PYG 02 cohort, officers are observed conditionally 
on their not having attrited after failing their first or second boards. In the PYG 03 
cohort, officers are observed conditionally on their not having attrited after their first 


board. 


Table 1 illustrates the truncation and censoring patterns of the data used in the 
thesis research. Table entries of “T” signify that data are truncated (not observed) at the 


indicated value of Y. Table entries of “C” signify that data are censored at the indicated 



































value of Y. 
Selected Attrite Attrite Attrite 
on first after first | Selected on after Selected on | after third 
look look second look | second look | third look look 
Cohort (Y = 1) (Y =2) (Y = 3) (Y = 4) (Y =5) (Y = 6) 
PYG 02 T T 
PYG 03 T C C C 
PYG 04 C C C C C 
Table 1. Truncation and Censoring Patterns of Data. 
4. Maximum Likelihood Estimation 


To estimate the MHLR model parameters in the presence of both truncation and 


censoring, the entire model is fit to the data simultaneously using maximum likelihood. 


This is equivalent to maximizing the following log-likelihood function: 
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LB,» Bys +> Bes) =L, (B,, B+» Bea) = (B,, By--- Bea) 
Le (By Bor Bos) =Slox {> ples x(0)| 


Ly (B,, Bos-- Bea) = Vlog} te ptt x, (oh 
i=l 
The log-likelihood is expressed as a censoring contribution (C) minus a truncation 
contribution (2). The set C, consists of all values represented by the outcome for officer 
i. If this officer succeeded on the second board, for example, then Y, =3 and C, consists 


of the single value 3. If this officer belonged to PYG 04 and failed to make his first 


board, then Y, >1 is right-censored and C, = {2,3,4,5,6}. Because all censoring is right- 
censoring in our case, C, consists either of a single number or as the set of all integers 
greater than or equal to a certain value. 

The set & consists of all outcomes that the officer was eligible to observe. If the 
officer belonged to the PYG 02 cohort, then & = {1,3,5,6}; ie., the outcomes Y, =2 and 
Y,=4 were truncated from this cohort. Similarly, officers in the PYG 03 cohort have 
J ={1,3,4,5,6}, because officers who will attrite after failing their second board (Y, = 4) 
are included in the sample. Officers in the PYG 04 cohort are unaffected by truncation, 
and & = {1,2,3,4,5,6} for these officers. 

The model parameters £,, £,,...,8,., enter into the log-likelihood function 
through the probabilities p(t; x,(t)). Using the inversion relationship together with the 


logit model for the hazard function, these probabilities are expressed as follows: 


exp(x, (1) B,) 





— -, 1<t<k-1 
[ [| t+ expe;B;) 
p(t; x,(t))=4 7" . 
= =, t=k 
[ [| [+ expe 8) 








jal 
The likelihood is maximized by taking all first and second partial derivatives of 


L(B,, B,,-.-.8,.,) with respect to the parameters and using a Newton-Raphson 
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algorithm. The function nlminb in S-Plus was used to implement a Newton-Raphson 
algorithm on minus the log-likelihood because nlminb attempts to locate the minimum, 


as opposed to the maximum, of the user-supplied function. Upon successful 
convergence, the maximum likelihood estimates (MLEs) B. | eae Be. were obtained. 
An estimated covariance matrix of the MLEs is obtained by inverting minus the Hessian 
of £(B,, B,..--,B,_,) evaluated at B. Boyes B..- Successful convergence ensures that 


the estimated covariance matrix is positive definite. Estimated standard errors of 
individual parameters are obtained as the square root of the corresponding diagonal 
element of the estimated covariance matrix. These and other concepts of maximum 


likelihood estimation are explained in Bickel and Doksum (2001). 


5. Hypothesis Testing Under the Hazards Model 

Under “regularity” conditions, maximum likelihood estimators are approximately 
unbiased and normally distributed in large samples (Bickel and Doksum, 2001). It can be 
verified that these conditions are satisfied by the hazards model. Based upon this 
property, individual parameter estimates are approximately normally distributed, centered 
on the true parameter values, with standard errors estimated as described above. 
Standardizing the estimated coefficients (dividing by their standard errors) gives a means 
of testing whether individual parameters are significantly different from zero. 
Standardized coefficients that exceed 1.96 in absolute value are significant at the 


a =.05 test level. 


To test whether sets of coefficients are equal to zero, a likelihood ratio test (LRT) 
is used (Montgomery, 2001). First, a “full model” is fit to the data that includes the 
parameters in question. Second, a “reduced model” is fit, containing all parameters of the 
full model except those that are being tested for equality to zero. Maximum likelihood is 


used to fit both the full and the reduced model. The LRT statistic is given by 
i= ops (full model) — Z (reduced model) | 


Under the null hypothesis that the parameters in question (those appearing in the 


full model but not in the reduced model) are equal to zero, the test statistic y has 
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approximately a chi-square distribution with degrees of freedom equal to the number of 
parameters that are being tested. The null hypothesis is rejected at level a if y exceeds 
the 1—a@ quantile of the chi-square distribution with the corresponding degrees of 
freedom. 

6. Data Used in Model Estimation 

There are 616 total observations in the data set that was available for analysis. In 
26 cases, data were missing in at least one key variable, and these observations were 


removed, leaving 590 usable observations for estimation under the MHLR model. 
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IV. ANALYSIS AND RESULTS 


This chapter presents the results applying the MHLR model to the 2002-2004 
SWO XO selection-board data. Nonparametric statistical methods are used to provide 
additional insights. 
its ASSUMPTIONS FOR DATA ANALYSIS 

A basic assumption that underlies the analysis presented in this chapter is that the 
data can be regarded as equivalent to a random sample taken from a relevant population, 
in this case all officers who are subject to the SWO XO selection process. With the 
exception of officers who may have attrited from the process in the PYG 02 and PYG 03 
cohorts, the data represent an exhaustive sample of the three cohort years that are studied. 
The assumption of randomness is that the selection process is stationary with respect to 
time, and that the cohorts included in the thesis research are representative of all officers 


who have been and will be subjected to this selection process. 


Throughout the thesis research, it is assumed that each observation may be treated 
as an independent random sample from the population of interest, without sampling bias. 


B. FITTING THE MULTISTAGE HAZARDS LOGISTIC REGRESSION 
(MHLR) MODEL 


The base number of observations is 590, which is the number of officers from the 
PYG 02 through PYG 04 cohorts with complete data in all variables, (as defined in 
Chapter III). Further reductions in the data used for fitting the MHLR model must also 
be made for reasons that are discussed below. In order to fit the model a subset of the 
explanatory variables is then identified which contribute significantly to the estimation of 
probabilities related to outcomes of the XO selection process. 
i Be Data Excluded from the Analysis 
To fit the MHLR model to the data, it is necessary to remove certain groups of 
data from the analysis. There groups are described below: 
e Thirteen female SWOs were eligible for selection to XO in the PYG 02 
through PYG 03 cohorts. These officers were excluded from model fitting 
for several reasons. First, it was desired not to assume that these officers 


were similar to male officers in the absence of evidence to that effect. 
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Second, there are not enough data from these thirteen officers to treat them 
separately in the MHLR model. And, as noted in Chapter II, Navy policy 
specifically states that board members must ensure candidates are not 
disadvantaged due to gender. Therefore, these thirteen female SWOs are 
not included in the model. Six of the thirteen female SWOs (46 percent) 
were selected for XO, compared with 258 of the 577 male SWOs (45 


percent). 


* All officers receive a promotion recommendation on their FITREP. Any 
rating of “significant problems” or “progressing” is considered to be an 
adverse report. Receiving either of these marks significantly reduces an 
officer’s chance of selection. Sixteen officers received an adverse report 
and none was selected. Due to this strongly negative effect, these 16 


officers are not included in the analysis. 


e Command at sea at the O3 pay grade is considered to be a significant 
achievement and is a career milestone. Twenty-two officers had 
command at sea at the O3 pay grade, and all of them were selected for 
XO. Due to this strongly positive effect, these 22 officers are not included 


in the analysis. 


After removing these three classes of officers, 540 officers remain in the 
data that are used in the analysis. 


Zz Choice of Explanatory Variables 


The following table defines the explanatory variables used in the MHLR model. 
Not all variables were used in every analysis. The models considered in this thesis are as 


follows: 


e The full model contains 34 explanatory variables extracted from the OAIS 
and EMPRS data. These explanatory variables are used over five 


regressions in the MHLR model. 
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e Four reduced models, each containing a subset of the 34 explanatory 
variables in the full model, are defined. The reduced models are used to 
test the significance of explanatory variables in estimating probabilities 


related to the XO selection process. 


Detailed descriptions of the models considered in this thesis are given in the 











appendix. 
Variable Description 

DC Whether an officer has completed a tour in Washington DC or 
the Bureau of Naval Personnel (1 = Yes, 0 = No). 
Whether an officer has completed a Master’s degree (1 = Yes, 

MA 
0 = No). 

JPME Whether an officer has completed Joint Professional Military 


Education Phase I (1 = Yes, 0 = No). 
NUKE Whether an officer is nuclear trained (1 = Yes, 0 = No). 
EAST Whether an officer has completed a DH tour in a billet on the 
East Coast (1 = Yes, 0 = No). 
Whether an officer has completed a DH tour in a billet on the 
West Coast (1 = Yes, 0 = No). 
LEFT1, LEFT2, Whether an officer has promotion recommendation that 

LEFT3 decreased over time on his FITREPs (1 = Yes, 0 = No). 
A score based on FITREP ratings received at the O3 pay 
O3SCORE grade. Higher scores imply that the officer received higher- 
than-average ratings than peers at this pay grade. 
Whether an officer was an early roller (go directly from 
pene ee DIVO tours to DH tours) (1 = Yes, 0 = No). 

PRIOR Whether an officer was prior enlisted (1 = Yes, 0 = No). 
RD Whether an officer completed three department head tours 

oy SUEAEEE afloat (1 = Yes, 0 = No). 
Table 2. Explanatory Variables Used in the Models. 








WEST 


























The following gives a more detailed explanation of how the O3SCORE score is 
calculated: 

Step 1 Determine cutoff date of FITREPs. The cutoff date is 31 March in the 
year than a SWO is first reviewed by a XO selection board. O3SCORE 
uses all FITREP information up to the cutoff date for the time that the 
SWO was at the O3 pay grade. 
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Step 2 For each FITREP rating at the O3 pay grade up to the cutoff date, divide 
by the average FITREP rating issued by the same reporting senior officer. 
Exclude any FITREP for which the officer received a rating of “NOB” 
(Not Observed). 


Step 3 Take the minimum of the ratios described in Step 2. If there was only one 
reporting senior officer for the SWO at the O3 pay grade, only that ratio is 


used. 


An O3SCORE equal to one implies that the candidate is at the average rating for a 
senior reporting officer at the O3 pay grade. Values less than one suggest lower-than- 


average ratings, and values greater than one suggest higher-than-average ratings. 


3. Software for Model Fitting 

The data were analyzed using software that was written in the S-Plus® 7.0.0 
(Insightful Corp.) statistical programming language. S-Plus functions were written for 
the likelihood function of the MHLR model under censoring and truncation, and for the 
first and second derivatives of the likelihood function. The nlminb function in S-Plus 
used these functions to maximize the likelihood, which in turn provided maximum 
likelihood estimates of the MHLR model parameters. The nlminb function uses a 
Newton-Raphson algorithm to find the maximum of the likelihood function. Estimated 
standard errors of the model parameter estimates were obtained from the second 


derivative matrix (Hessian) that was output from nlminb. 


Microsoft Excel 2003® (Microsoft Corp.) was also used for the creation of graphs 
in the thesis. 

4. Testing for the Significance of Truncation 

As explained in Chapter III, truncation occurs because officers who attrited from 
the XO selection process between boards are not observed. The MHLR model includes 
probabilities for attrition after the first board (Y = 2) and after the second board (Y = 4). 
Including these probabilities in the model may present numerical problems, due to the 
inability of the model to distinguish the occurrence of truncation from the data in the 
presence of truncation and censoring. This would result in the model attempting to 


assign probabilities of zero to these events, which can happen only if the corresponding 
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parameters approach negative infinity. Although these probabilities cannot be equal to 
zero as a matter of principle, the model may not be able to detect a signal to that effect 
from the data. In essence, setting these probabilities equal to zero in the model is the 
same as treating the sample data as if truncation were not present. The issue is whether 


ignoring truncation has a significant effect on model estimation. 


In testing for the significance of truncation, a full model and a full model 
without truncation are defined. The full model has 34 parameters distributed over five 
regressions. Table 3 lists the explanatory variables appearing in the full model. The full 
model without truncation (which assigns probabilities of zero to the two attrition 
outcomes) has 32 parameters distributed over three regressions. Both models are fit to 
the data, and a likelihood ratio test (LRT) (Montgomery, 2001) is used to ascertain 
whether the full model (which includes attrition) is significantly better than the full model 
without truncation. Failure to reject the null hypothesis would suggest that there is no 


evidence that ignoring attrition reduces the quality of estimation. 


















































Regression 1 Regression 2 Regression 3 Regression 4 Regression 5 
1. CONST 13. CONST* | 14. CONST 24. CONST* | 25. CONST 
2. DC 15. MA 26. MA 
3. MA 16. JPME 27. JPME 
4. JPME 17. NUKE 28. NUKE 
5. NUKE 18. EAST 29. EAST 
6. EAST 19. WEST 30. WEST 
7. WEST 20. LEFT2 31. LEFT3 
8. LEFT1 21. O3SCORE 32. O3SCORE 
9. O3SCORE 22. PRIOR 33. PRIOR 
10. EARLY.ROLLER 23. 3°? DH AFLT 34. 3°? DH AFLT 
11. PRIOR 
12. 38° DH AFLT 








Table 3. Explanatory Variables Used in Full Model. 
The full model without truncation contains the same explanatory variables shown in 
the table, with the exception of Regression 2 and Regression 4 which are absent 
from the model without truncation. 
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Parameter estimates obtained from the full model and full model without 


truncation are presented in the Appendix. The p- value of the LRT is approximately 


equal to .746 and the null hypothesis cannot be rejected at thea =.05 test level.The null 
hypothesis that the probabilities of falling into categories Y =2 and Y =4 are both equal 
to 0 is not rejected because there is insufficient evidence from the data to the contrary. 
Further estimation, however, is based on the full MHLR model. 
C. TESTING FOR SIGNIFICANCE OF EXPLANATORY VARIABLES 

A series of reduced MHLR models are considered to test the significance of 
explanatory variables of interest. These models are compared to the full model defined 
above and LRTs are conducted to determine whether the explanatory variables in 
question are statistically significant. Four reduced models are considered. These model- 
fitting exercises suggest that, at an officer’s first review, DC, MA, and O3SCORE are 
significant factors in determining his probability of success. Given that an officer was 
not selected in his first review, his probability of success at the second review is most 
strongly influenced by EAST, WEST, and O3SCORE. Finally, given that an officer was 
not selected in his first or second review, his probability of success at the third (and final) 


review is most strongly influenced by NUKE and O3SCORE. 


Four reduced models were built that tested the significance of certain explanatory 
variables. The results of each reduced model are presented in the Appendix. Table 4 


summarizes the results of these tests. 









































Log- p- 

Model Parameters | likelihood | Deviance | Df | value 
Full 34 -429.3729 

Reduced without DC, MA, | 27 -444.5796 | 30.053 7 | 0.000 
and JPME 

Reduced without DC 33 -437.0581 | 15.370 1 | 0.000 
Reduced without MA 33 -434.4084 | 10.071 1 | 0.018 
Reduced without JPME 31 -432.9797 | 7.2136 1 | 0.065 

Table 4. Result of Test of Significance of Explanatory Variables 


Deviance is defined as two times the difference of the log-likelihood in both 


the full model and reduced model (Montgomery, 2001). 


e Reduced Model without DC, MA, and JPME Variables 
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This reduced model includes 27 parameters. It includes all of the explanatory 
variables from the full model except DC, MA, and JPME. These variables are all shore- 


duty-related variables that are assumed to be not achievable while on sea duty. 


The results of this model indicate that for all three reviews an officer can have, the 
O3 score is the single-most-important variable. This variable is an extremely positive 
variable toward the regressions, and in regressions one, three, and five (regressions which 


represent a selection board), the p - value is 0. 


This reduced model has a log-likelihood of -444.5796. The LRT of this model 
with respect to the full model defined earlier is equivalent to testing the following 


hypotheses: 
Ay: B, = B= B, = Bs = Bro = Bas = Boy =9 
H,: not H, 





The null hypothesis is equivalent to the statement that none of the seven 
explanatory variables in the full model involving DC, MA, and JPME affect an officer’s 
probability of selection to XO. The alternative hypothesis is the opposite statement, 
namely that at least one of these seven variables does affect the probability of selection. 
Assuming a model deviance chi-squared distribution with seven degrees of freedom, the 
p- value is approximately 0. Therefore, the null hypothesis can be rejected at the 
a@=.05 level which suggests that DC, MA, and JPME significantly contribute to the 
MHLR model. 

e Reduced Model without DC Variable 

This reduced model includes 33 parameters. It includes all of the explanatory 
variables from the full model except DC. The results of this model indicate that for all 
three reviews an officer can have, the O3 score is the single-most-important-variable. 
This variable is an extremely positive variable toward the regressions, and in regressions 
one, three, and five (regressions which represent a selection board), the p- value is 


approximately 0. 
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This reduced model has a log-likelihood of -437.0581. Assuming a chi-squared 
distribution with one df, the p- value is approximately 0. The null hypothesis can be 
rejected at the @ =.05 level, and it shows that DC is significant to the model. 

e Reduced Model without MA Variable 

This reduced model includes 31 parameters. It includes all of the explanatory 
variables from the full model except MA. The results of this model indicate that for all 
three reviews an officer can have, the O3 score is the single-most-important-variable. 
This variable is an extremely positive variable toward the regressions, and in regressions 


one, three, and five (regressions which represent a selection board), the p- value is 


approximately 0. 


This reduced model has a log-likelihood of -434.4084. Assuming a chi-squared 
distribution with three df, the p- value is approximately 0.018. The null hypothesis can 
be rejected at the @ =.05 level, and it shows that MA is significant to the model. 

e Reduced Model without JPME Variable 

This reduced model includes 31 parameters. It includes all of the explanatory 
variables from the full model excep: JPME. The results of this model indicate that for all 
three reviews an officer can have, the O3 score is the single-most-important-variable. 
This variable is an extremely positive variable toward the regressions, and in regressions 
one, three, and five (regressions which represent a selection board), the p- value is 
approximately 0. 

This reduced model has a log-likelihood of -432.9797. Assuming a chi-squared 
distribution with three df, the p- value is approximately 0.065. Although the null 
hypothesis cannot be rejected at the @=.05 level, it is rejected at the @=.10 level, 
which suggests that JPME is a marginally important factor in the model. 


C. Probabilities of selection 


The MHLR model can be used to predict whether an officer will be selected for 
XO. An officer is predicted to have an outcome (value of Y) that has the highest 
probability among the six possible outcomes. For example, if the highest probability is 


Y =1, a SWO is predicted to be selected on his first look. For officers whose data are 
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used in the thesis research, it is possible to compare these predictions to actual outcomes, 
provided that their outcomes are not censored. They include all officers from PYG 02, 
officers from PYG 03 who were selected on the first or second review, and officers from 
PYG 04 who were selected on the first review. A total of 309 officers belong to these 
groups. Figure 3 shows boxplots that describe the distributions of predicted probabilities 


for each of the actual outcomes among the 309 officers used in this analysis. 
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Figure 3. For Each Actual Outcome, a Box Plot Graphs the Predicted 
Probability Versus Actual Outcomes for Each Predicted Outcome 
State. 


A box plot can be read as follows. The lowest line in each column is the minimum 
value, the lower line of the box is the 50 percentile, the middle line in the box is the 
median (50" percentile), the upper line in the box is the ‘Si percentile, and the highest 
line is the maximum value. For example, among SWOs who were selected in their first 
review (Y = 1), the estimated probabilities of selection in the first review vary about a 
median of approximately .40. Moreover, these probabilities are higher than for any of the 
other outcomes. A similar pattern also holds for officers who were selected in the second 


and third reviews, and officers who were not selected. Generally, officers who were 
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selected in one of the three reviews had low probabilities of not being selected, and 


officers who were not selected had not being selected as their most likely outcome. 


The following table shows the prediction accuracy of the MHLR model. All 
selection outcomes (Y =1,3, or 5) are combined and contrasted with the non-selection 
outcome (Y =6), again using the 309 officers described above. An officer is predicted to 
be selected if the estimated probability of selection exceeds the estimated probability of 
non-selection. Similarly, an officer is predicted not to be selected if this outcome has a 
higher estimated probability than that of being selected. The model correctly classifies 
approximately 76 percent of officers who were actually selected, and 79 percent of 


officers who were not selected. 


























Correct 
Description Selection | Total Observations | Classification 
Predicted board selection 187 247 75.71% 
Predicted board non-selection | 13 62 719.0% 


Table 5. Accuracy of the Full Model. 


D. MANTEL-HAENSZEL TEST 

1. Methodology 

A nonparametric approach was taken to explore whether having certain variables 
affects selection. Due to truncation and right censoring, only an officer’s first board 
review can be analyzed. A series of upper-tailed Mantel-Haenszel tests (Conover, 1999) 
were conducted to assess dependence among variables conditional on the year group of 


an officer. 


The Mantel-Haenszel test is used for testing independence of two dichotomous 
variables conditional on a third (discrete) variable. The data are cross-classified 
according to the three variables, resulting in a series of k two-by-two contingency tables, 
where k is the number of levels of the conditioning variable. The i" level of the 
conditioning variable is represented in Table 6: 


Column 1 Column 2 





Row 1 | x, r—X, 


a 





Row 2 | ¢,-x, Neh eeea| N= 


= 
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The total number of observations in the i” 
table is N,. Cells in the table show counts in 


the given row and column. The column total 
in the i” table is c,. The row total of the i” 


table is r. 


L 


Table 6. Representation of Contingency Tables for Use in the Mantel-Haenszel 
Test. (Conover, 1999) 


Each observation is classified into exactly one cell for each contingency table. 
Further, the row and column totals are fixed, not random. The test statistic is shown as 


follows: 





T= 4% — > Ge) iN, 


= Ldn, —r,)(N, —c,)/N; 





The distribution of T is approximately standard normal when the null hypothesis of 
conditional independence between rows and columns is true. Specifically, let p,, be the 
probability of an observation in Row 1 being classified into Columnl, in the i” 
contingency table, and let p,, be the probability of an observation in Row 2 being 


classified into Column1. For an upper-tail test, the following hypothesis is tested: 
Hy: Pi; < Po;, forall i =1,...,k 


H,: p,,2 p>, for alli, and p,, > p,, for at least one j. 
The null hypothesis is rejected at levela if T is greater than the 1—a@ quantile of the 
standard normal distribution. Equivalently, the null hypothesis is rejected if the p - value 
(i.e., the probability of a standard normal random variable being greater than the observed 
value of 7) is less thang . 
In the following analyses, “Row” refers to any of several dichotomous 
explanatory variables including MA, JPME, DC, EAST, and WEST; “Column” refers to 


the outcome of being selected or not selected at the first review. The conditioning 
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variable is the year-group of an officer, which takes on the values PYG 02, PYG 03, or 
PYG 04 (k = 3). A total of 540 officers (those used in fitting the MHLR model) are used 
in the Mantel-Haenszel tests. 

2. Mantel-Haenszel Test Results 

e MA versus Selection 

The Mantel-Haensel test statistic is T = .2924 with a p - value of .5887. This p- 
value suggests that there is insufficient evidence to reject the null hypothesis at a 
significance level of a =.05. 

e DC versus Selection 

The Mantel-Haenszel Test produced a value of 19.8569 with a p- value of 0. 
This p - value suggests strong evidence against the null hypothesis at a significance level 
of a=.05. 

e JPME versus Selection 

The Mantel-Haenszel Test produced a value of 5.7421 with a p - value of .0166. 
This p - value suggests strong evidence against the null hypothesis at a significance level 
of a=.05. 

e EAST versus Selection 

The Mantel-Haenszel Test produced a value of .3085 with a p- value of .5786. 
This p- value suggests that there is no evidence from the known data to reject the null 
hypothesis at a significance level of a =.05. 

e WEST versus Selection 

The Mantel-Haenszel Test produced a value of .3044 with a p- value of .5811. 
This p - value suggests that there is no evidence from the available data to reject the null 


hypothesis at a significance level of a =.05. 
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Table 7 summarizes the results of the Mantel-Haenszel tests: 






































Variable | df | Value p - value Conclusion 

MA 1 0.2924 | 0.589 Do Not Reject Null Hypothesis 
DC 1 | 19.8569 | 0.000 Reject Null Hypothesis 

JPME 1 5.7421 0.0166 | Reject Null Hypothesis 

EAST 1 0.3085 | 0.5786 | Do Not Reject Null Hypothesis 
WEST 1 0.3044 | 0.5811 | Do Not Reject Null Hypothesis 





Table 7. Results of Upper-Tailed Mantel-Haenszel Tests at the a@ =.05 Test Level. 


E. CONDITIONING ON 03 SCORE 

i, Methodology 

Results of the Mantel-Haenszel suggest that having a Master’s Degree, having 
completed an East Coast DH tour, or having completed a West Coast DH tour does not 
affect a candidate’s probability of selection to XO, but having completed a Washington 
DC or BUPERS tour and having completed JPME has a positive effect on selection. 
Based on the results of the Manel-Haenszel test and the results of the MHLR model, the 
O3SCORE remains the most important factor contributing to selection or non-selection to 


XO. 


To explore the effects of dichotomous variables conditional on O3SCORE, the 
results of each officer’s first review are cross-tabulated. The first review is used because 
it is unaffected by censoring in all three cohort groups. The conditioning event is that 
O3SCORE is less than or equal to s, where s varies between .88 and 1.14. Table 8 is an 


example of the table used. 














PYG 
Selected | Not Selected | Total Percent Selected 
Officer with Trait of Concern x a4 x(x+y) 
Officer without Trait of Concern | a b al(at+b) 




















Table 8. Table Used to Determine the Proportion Selected Conditioned Upon the 


O3SCORE. 
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The variable x represents the total number of officers selected with the trait of 
concern. The variable y represents the total number of officers not selected with the trait 
of concern. The variable a represents the total number of officers selected without the 
trait of concern. The variable b represents the total number of officers not selected 
without the trait of concern. The total percent selected column is derived by dividing the 
number of officers selected by the sum of the number selected and not selected in each 
group. 

This test is cumulative in the sense that the O3SCORE is variable and as the 
O3SCORE gets larger, a larger number of officers are included. Conditioning on 


O3SCORE less than or equal to s in the range [.88, 1.14] was performed in increments of 
02. 


2 Results of Conditioning on O3SCORE 

e Washington DC or BUPERS Tour versus Selection 

Conditioning on O3SCORE as described above, having completed a tour in 
Washington DC or BUPERS improves the chances of selection to XO. This result is 
consistent with the findings of the MHLR model that having completed a Washington 
DC or BUPERS tour is a positive contributor to the MHLR model. It must be noted that 
an unfavorable O3SCORE is a difficult obstacle for a SWO to overcome. Figures 4 


through 6 show the proportion selected for XO broken down by cohort. 
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Figure 4. The Proportion Selected to XO Versus O3SCORE of the Officer for 


the PYG 02 Cohort Group and the DC Variable. 
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so°o 
an N 


0.4 








So 9 
wp wo 








Proportion Selected 
° 


oO 











O3 Score 





—e— PYG 03 without DC —s— PYG 03 with DC 














0.88 0.9 0.92 0.94 0.96 0.98 1 1.02 1.04 1.06 1.08 1.1 1.12 1.14 





Figure 5. The Proportion Selected to XO Versus O3SCORE of the Officer for 


the PYG 03 Cohort Group and the DC Variable. 
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Figure 6. The Proportion Selected to XO Versus the O3SCORE of the Officer 
for the PYG 04 Cohort Group and the DC Variable. 


Figures 4 through 6 reveal useful insights into the importance of O3SCORE to an 
officer’s chance of selection. An officer with a low O3SCORE has very little chance of 
being selected without having completed a Washington DC or BUPERS tour. Completed 
such a tour significantly increases his chance of selection regardless of O3SCORE. 

e The Effect of a Master’s Degree Conditioned on O3SCORE 

Conditioning on the O3SCORE, having completed a Master’s Degree does not 
improve the chance of selection to XO. This result is consistent with the findings of the 
MHLR model that having a Master’s Degree does not contribute to selection. Figures 7 


through 9 show the proportion selected by PYG conditioning of O3SCORE. 
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Figure 7. The Proportion Selected to XO Versus O3SCORE of the Officer for 
the PYG 02 Cohort Group and the MA Variable. 
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Figure 8. 


The Proportion Selected to XO Versus O3SCORE of the Officer for 
the PYG 03 Cohort Group and the MA Variable. 
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Figure 9. The Proportion Selected to XO Versus the O3SCORE of the Officer 
for the PYG 04 Cohort Group and the MA Variable. 


Figured 7 through 9 demonstrate how having a Master’s Degree is not that 
influential in determining whether an officer will be selected for XO. In all three cohort 
groups, of the officers with a higher O3SCORE, a higher proportion of officers without 
Master’s Degrees were selected than those with Master’s Degrees. This trend leads to the 
conclusion that having a Master’s Degree does not increase the chances of selection and 
that the O3SCORE is more important between these two variables. 

e The Effect of JPME Conditioned on O3SCORE 

Conditioning on the O3SCORE, having completed JPME does improve the 
chance of selection to XO. This result is consistent with the findings of the MHLR 
model that having completed JPME is a positive contributor to the MHLR model. 
Figures 10 through 12show the proportion selected by PYG. 
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Figure 10. The Proportion Selected to XO Versus O3SCORE of the Officer for 
the PYG 02 Cohort Group and the JPME Variable. 
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Figure 11. The Proportion Selected to XO Versus O3SCORE of the Officer for 
the PYG 03 Cohort Group and the JPME Variable. 
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Figure 12. The Proportion Selected to XO Versus O3SCORE of the Officer for 
the PYG 04 Cohort Group and the JPME Variable. 


The above graphs suggest that having completed JPME is a contributing factor to 
selection. In PYG 02 and PYG 03, having completed JPME significantly increases the 
probability of selection regardless of O3SCORE. 
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Vv. CONCLUSIONS 


This thesis attempts to answer four research questions. These research questions 
allow for quantification of the selection board process. They allow for an officer to 
understand what his chances are for selection to XO and to be able to best manage his 
career. These questions and their resulting answers appear below. 

A. RESEARCH QUESTIONS 
e How Important is Performance at Sea to Being Selected to Executive 


Officer? 


The results show that both shore and sea duty variables are important. However, 
an officer’s DH FITREP scores have the largest influence on selection to XO. 
Additionally, SWOs who serve in command at the O3 pay grade select at a higher rate 
than do other officers. These officers are given responsibility early in their career and 
their performance during this challenging sea duty job sets them apart from their peers. 

e What is the Probability that a Surface Warfare Officer Will be Selected for 

Executive Officer by the First Selection Board, the Second Selection Board, 


the Third Selection Board, or that he will not be Selected? 


The MHLR model described in this thesis is able to predict whether an officer 
will or will not be selected for XO. This model can be applied to past, present, or future 
data sets assuming the selection board process stays constant. For the PYG 02, 03, and 
04 cohort groups, an officer that is predicted to select for XO will select 75.7 percent of 
the time and an officer that is predicted to fail to select for XO will fail to select 79 
percent of the time. 

e Which Factors in a Surface Warfare Officer’s Career Profile Improve his 


Chances for Selection to Executive Officer? 


It has been established through this thesis that the single most important variable 
affecting selection for XO is an officer’s DH FITREP scores. In other words, superior 
performance at sea as a DH (greater than 1.0 if using the model in this thesis) has the 


largest influence on an officer’s potential selection to XO. Variables such as having 
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completed a tour in Washington DC or BUPERS or having completed a Master’s Degree 
are significant, but sustained superior performance at sea (DH FITREP scores) carries the 
most weight. 

e Which Factors in a Surface Warfare Officer’s Career Profile Reduce his 


Chances for Selection to Executive Officer? 


Fitness Report scores are key to selection to XO. An officer with low FITREP 
scores (below 1.0 if using the model in this thesis) will reduce his chances for selection to 
XO. It is difficult to recover from low FITREP scores regardless of any other jobs or 
education an officer has completed. Every officer who was rated below “‘promotable” on 
a FITREP failed to select for XO. 

B. FUTURE RESEARCH 

There are many additional research opportunities that can evolve from this thesis. 
First and foremost is data reliability. Due to unavailability of data, there is truncation. If 
the information of all the officers at the beginning of the first board review were 
available, the estimates of the model would be significantly improved. In other words, 
reworking the model without truncation would improve the estimates of the model. 
Additionally, if data were available for other boards, the amount of right censoring would 


decrease which again would improve the estimates of the model. 


The results of this research indicate that an officer only is given one chance to 
perform. If an officer’s performance causes him to receive a mark below “Promotable” 
at anytime after reaching the O3 pay grade, it will cause him to never be competitive for 
selection again. An interesting research topic would be to study these officers to see if 
they are still suitable for XO even if they made a mistake in the past that caused them to 


receive a mark below “Promotable.” 


The MHLR model is programmed and implemented in the S-Plus® 7.0.0 
(Insightful Corp.) statistical programming language. It would be desirable to have this 
model made available in other software platforms that the SWO community could use on 
a regular basis. Using predicted probabilities obtained from the MHLR model, the SWO 


community could optimize the assignment of mid-grade officers. 
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APPENDIX 














Log- p- 
Model Parameters | likelihood | Deviance | Df | value 
Full 34 -429.3729 
Full Model without Truncation 27 -429.6658 | .5858 2 | 0.746 



































































































































Table 9. Estimates from the Full Model and Full Model without Truncation. 
Regression | Variable Name | Coefficient | Standard Std p - value 
Error Coefficient 
1 CONST -12.85 1.99 -6.46 0.00 
1 DC 1.40 0.36 3.88 0.00 
1 MA -0.63 0.23 -2.76 0.01 
1 JPME 0.73 0.30 2.42 0.02 
1 NUKE 0.18 0.34 0.53 0.59 
1 EAST -0.44 0.34 -1.30 0.19 
1 WEST -0.36 0.33 -1.09 0.28 
1 LEFT1 -0.16 0.41 -0.40 0.69 
1 O3SCORE 13.23 2.03 6.53 0.00 
1 EARLY.ROLLER 2.10 0.72 2.91 0.00 
1 PRIOR -0.38 0.23 -1.67 0.10 
i 38° DH AFLT -0.14 0.67 -0.20 0.84 
2 CONST -2.47 3.17 -0.78 0.44 
3 CONST -11.56 2.80 -4.13 0.00 
3 MA 0.05 0.34 0.14 0.89 
3 JPME 0.57 0.49 1.16 0.25 
3 NUKE -0.59 0.64 -0.91 0.36 
3 EAST -0.95 0.55 -1.73 0.08 
3 WEST -1.15 0.51 -2.26 0.02 
3 LEFT2 -0.51 0.56 -0.91 0.36 
3 O3SCORE 12.76 2.95 4.32 0.00 
3 PRIOR -0.43 0.35 -1.22 0.22 
3 38° DH AFLT -0.17 0.71 -0.24 0.81 
4 CONST -1.88 1.94 -0.97 0.33 
5 CONST -11.66 4.09 -2.85 0.00 
5 MA 0.82 0.55 1.48 0.14 
5 JPME 0.50 0.79 0.63 0.53 
5 NUKE 2.85 1.21 2.35 0.02 
5 EAST 0.07 0.73 0.10 0.92 
5 WEST 0.17 0.64 0.26 0.80 
5 LEFT3 -0.41 0.71 -0.58 0.56 
5 O3SCORE 10.97 4.40 2.50 0.01 
5 PRIOR 0.39 0.57 0.68 0.49 
5 38? DH AFLT 0.63 1.06 0.59 0.56 
Table 10. Full Model Used in Testing Significance of Explanatory Variables. 
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Regression | Variable Name _ | Coefficient | Standard Standard p- 
Error Coefficient value 
1 CONST -12.05 1.89 -6.38 0.00 
1 NUKE 0.09 0.33 0.28 0.78 
1 EAST -0.42 0.33 -1.29 0.20 
it WEST -0.33 0.32 -1.04 0.30 
1 LEFT1 -0.17 0.41 -0.42 0.67 
1 O3SCORE 12.19 1.89 6.44 0.00 
1 EARLY.ROLLER 2.27 0.70 3.26 0.00 
1 PRIOR -0.38 0.22 -1.71 0.09 
1 38? DH AFLT -0.33 0.66 -0.50 0.62 
2 CONST -1.47 1.28 -1.15 0.25 
3 CONST -11.67 2275 -4.24 0.00 
3 NUKE -0.61 0.64 -0.96 0.33 
3 EAST -0.94 0.55 -1.72 0.08 
3 WEST -1.16 0.51 -2.29 0.02 
3 LEFT2 -0.57 0.56 -1.02 0.31 
3 O3SCORE 12.97 2.86 4.54 0.00 
3 PRIOR -0.49 0.35 -1.41 0.16 
3 38° DH AFLT -0.10 0.71 -0.15 0.88 
4 CONST -1.45 1.29 -1.12 0.26 
5 CONST -12.48 4.17 -2.99 0.00 
5 NUKE 2.70 1.16 2.33 0.02 
5 EAST 0.13 0.70 0.18 0.85 
5 WEST 0.17 0.61 0.29 0.77 
5 LEFT3 -0.31 0.70 -0.44 0.66 
5 O3SCORE 12.46 4.39 2.84 0.00 
5 PRIOR 0.29 0.55 0.52 0.60 
5 38> DH AFLT 0.76 1.04 0.73 0.46 


























Table 11. Reduced Model with no MA, JPME, and DC Used in Testing Significance 
of Explanatory Variables. 
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Regression | Variable Name _ | Coefficient | Standard Standard p- 
Error Coefficient value 
1 CONST -13.42 1.93 -6.95 0.00 
1 MA -0.62 0.22 -2.88 0.00 
1 JPME 0.81 0.28 2.88 0.00 
it NUKE 0.00 0.33 0.00 1.00 
1 EAST -0.28 0.31 -0.91 0.36 
1 WEST -0.23 0.30 -0.76 0.45 
1 LEFT1 -0.22 0.39 -0.56 0.58 
1 O3SCORE 13.76 1.96 7.00 0.00 
1 EARLY.ROLLER 2.13 0.71 3.00 0.00 
if PRIOR -0.38 0.22 -1.74 0.08 
1 38” DH AFLT -0.47 0.67 -0.70 0.49 
2 CONST -1.02 0.89 -1.14 0.25 
3 CONST -11.93 2.79 -4.28 0.00 
3 MA 0.01 0.33 0.02 0.98 
3 JPME 0.55 0.49 1.12 0.26 
3 NUKE -0.64 0.64 -1.00 0.32 
3 EAST -0.95 0.55 -1.73 0.08 
3 WEST -1.23 0.51 2A? 0.02 
3 LEFT2 -0.37 0.53 -0.70 0.48 
3 O3SCORE 13.26 2.94 4.51 0.00 
3 PRIOR -0.41 0.34 -1.19 0.24 
3 38° DH AFLT -0.19 0.71 “027 0.79 
4 CONST -2.13 2.34 -0.91 0.36 
5 CONST -11.66 4.09 -2.85 0.00 
S) MA 0.82 0.55 1.48 0.14 
5 JPME 0.50 0.79 0.63 0.53 
5 NUKE 2.85 eal 2.35 0.02 
5 EAST 0.07 0.73 0.10 0.92 
5 WEST 0.17 0.64 0.26 0.80 
5 LEFT3 -0.41 0.71 -0.58 0.56 
5 O3SCORE 10.97 4.40 2.50 0.01 
5 PRIOR 0.39 0.57 0.68 0.49 
5 38? DH AFLT 0.63 1.06 0.59 0.56 

















Table 12. Reduced Model with no DC in Testing Significance of Explanatory 
Variables. 
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Regression | Variable Name _ | Coefficient | Standard Standard p- 
Error Coefficient value 
1 CONST 2.12 1.93 -6.28 0.00 
1 DC 1.37 0.36 3.83 0.00 
1 JPME 0.60 0.30 2.03 0.04 
1 NUKE 0.16 0.33 0.49 0.62 
1 EAST -0.43 0.34 107 0.20 
1 WEST -0.32 0.32 -1.00 0.32 
1 LEFT1 -0.10 0.41 -0.24 0.81 
1 O3SCORE 12.10 1.94 6.24 0.00 
1 EARLY.ROLLER | 2.23 0.70 3.18 0.00 
1 PRIOR -0.32 0.23 -1.42 0.16 
1 3°? DH AFLT -0.16 0.67 -0.24 0.81 
2 CONST -2.62 3.63 20:79 0.47 
3 CONST -11.58 2.76 -4.20 0.00 
3 JPME 0.59 0.49 1.21 0.23 
3 NUKE -0.58 0.64 -0.91 0.37 
3 EAST -0.95 0.55 -1.74 0.08 
3 WEST -1.15 0.51 3.97 0.02 
3 LEFT2 -0.51 0.56 -0.90 0.37 
3 O3SCORE 12.84 2.87 4.48 0.00 
3 PRIOR -0.43 0.35 41.01 0.23 
3 3°? DH AFLT -0.18 0.71 -0.25 0.81 
4 CONST -2.26 2.70 -0.84 0.40 
5 CONST -12.65 4.20 -3.01 0.00 
5 JPME 0.61 0.76 0.80 0.42 
5 NUKE 2.81 1.17 2.39 0.02 
5 EAST 0.06 0.71 0.08 0.93 
5 WEST 0.09 0.62 0.15 0.88 
5 LEFT3 -0.35 0.71 -0.49 0.63 
5 O3SCORE 12.62 4.43 2.85 0.00 
5 PRIOR 0.37 0.56 0.65 0.51 
5 38? DH AFLT 0.79 1.05 0.75 0.45 





Table 13. Reduced Model with no MA Used in Testing Significance of Explanatory 
Variables. 
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Regression | Variable Name _ | Coefficient | Standard Standard p- 
Error Coefficient value 
1 CONST -12.92 1.97 -6.54 0.00 
1 DC 1.36 0.36 3.80 0.00 
1 MA -0.55 0.22 -2.47 0.01 
it NUKE 0.18 0.33 0.54 0.59 
1 EAST -0.45 0.34 -1.34 0.18 
1 WEST -0.36 0.33 -1.11 0.27 
1 LEFT1 -0.21 0.41 -0.51 0.61 
1 O3SCORE 13.37 2.01 6.64 0.00 
1 EARLY.ROLLER 2.05 0.72 2.87 0.00 
if PRIOR -0.41 0.23 -1.80 0.07 
1 38? DH AFLT -0.10 0.67 -0.16 0.88 
2 CONST -2.40 2.97 -0.81 0.42 
3 CONST -11.59 2.78 -4.17 0.00 
3 MA 0.07 0.33 0.22 0.82 
3 NUKE -0.62 0.64 -0.97 0.33 
3 EAST -0.93 0.55 -1.70 0.09 
3 WEST -1.16 0.51 -2.28 0.02 
3 LEFT2 -0.57 0.56 -1.02 0.31 
3 O3SCORE 12.81 2.94 4.36 0.00 
3 PRIOR -0.49 0.35 -1.40 0.16 
3 38? DH AFLT Final 0.71 -0.15 0.88 
4 CONST -1.28 1.15 -1.12 0.26 
5 CONST -11.55 4.07 -2.84 0.00 
5 MA 0.85 0.55 1.55 0.12 
=) NUKE 2.76 1.20 2.30 0.02 
5 EAST 0.14 0.72 0.20 0.84 
5 WEST 0.24 0.62 0.38 0.71 
5 LEFT3 -0.39 0.71 -0.56 0.58 
5 O3SCORE 10.83 4.37 2.48 0.01 
5 PRIOR 0.33 0.56 0.59 0.55 
5 38? DH AFLT 0.60 1.06 0.57 0.57 























Table 14. Reduced Model with no JPME Used in Testing Significance of Explanatory 
Variables. 
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